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Abstract 

This paper presents a model of an interactive mode for instructional 
research and development that might yield both curriculum products and 
scientific knowledge concerning learning and instruction. Curriculum de- 
velopment is discussed as an applied science, and as an art, using specific 
research and development efforts conducted at LRDC to highlight the pres- 
ent and potential character of the interactive model. Consideration is 
i;iven to the roles of task analysis, diagnosis of learner characteristics, 
design of curricula and learning environments, and assessment and evalu- 
ation in the development of instructional programs. 
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THE SCIENCE AND APT OF CURRICULUM DESIGN 

Lauren-B. Resnick 

Learning Research and Development Center 
University of Pittsburgh 

Education, broadly viewed, is any process that results in expanded 
knowledge and competerce for those who engage in it. Typically, we think 
of education as goinj, on in certain institutions- - schools, colleges, train- 
ing centers --but in fact the educational process is not Uniited to such 
institutions. It can. and does, occur in virtually all life encounters, in 
unplanned as well as planned ways. 

Curriculum, by Contrast, constitutes a planned intervention for the 
purpose of education. A curriculum can be thought of as a series of activi- 
ties explicitly designed to change the knowledge and competence of those 
who engage in it. Whenever an educational experience is planned, when- 
ever explicit efforts are made to optimize learning and development, a 
curriculum is being designed. The curriculum may be tight or fluid ia 
style: it may specify activities in great detail or only in general outline; 
the inatructor may control most moves or much may be left to learners. 
Whatever the particular strategy or ideology of edt^cation employed, it is 
appropriate to speak of a curriculum whenever education is not simply left 
to chance. 

In som,e societies, the largest part of the individual's education is 
"informal" in the sense that it takes place in the context of normal work 
or social activity (.Scribner & Cole. 1973). While some verbal instruction 
or demonstration may be offered, this is always done in a concrete con- 
text, thus relatively little linguistic and symbolic activity is required, and 
direct application to culturally relevant tasks is assumed. In a society 
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such as ours, however, which demands ^omplex symbolic abilities and 
flexibility in applying them across a ra^nge of contexts, learning would be 
too inefficient without some planned e^ucat» tnal interventiur. Completely 
"informal" learning would be too context -tied to permit the flexible appli- 
cations demanded in an industrialized environment. Environments explicit- 
ly designed for learning are thery^ore necessary in our society. Such en- 
vironments are designed to convey both general cultural skilli> such as 
literacy, mathematics, or reasoning, and specific knowledge or technical 
competence. They are also, /in principle, designed to convey such skills 
and knowledge reliably, i.e/, in a manner that assur&s individuals the 
possibility of learning. TVe design of environments for learning, and of 
components of these environments, is the task of curriculum design. 



Curriculum Design as Applied Scie nce 

Curriculum design can be viewed as a special case of engineering or 
applied science. So viewed, it becomes pertinent to ask what the scientific 
basis for design in the curriculum domain is, and how that scientific base 
interacts with the practice of design. The dominant view of the relation- 
ship between scienci; and social applications holds that science produces 
general principles cr laws, and these are studied by individuals- -enginee rs- 
who apply the laws to practical problems. According to this model, which 
is schematized in Figure 1, the flow of information is unidirectional. 
Scientists provide information to technologists, but technologists do not 
provide information to scientists. Furthermore, technologists do not 
even participate actively in providing questions for scientists to work on. 
Rather, scientific activity responds primarily to questions generated by 
previous theoretical research. Thus, if the scientist ever leaves the 
laboratory, it is to give "advice" to engineers or to translate his findings 
Into forms understandable to a wider audience. The scientist is not. 
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according to this view, directly responsible for what is done with his 
principlea. He is not expected to study directly the social outcomes of 
his work; he is responsible only for producing knowledge. 

Probably no one acttially engaged in scientific or technological work 
has ever really believed in the unidirectional model of information flow. 
When wartime exigencies demanded technological applications, physical 
scientists organized themselves into special working groups, focusing on 
technological outpxits yet yielding major new scientific knowledge. The hot 
and cold war conditions of the 1940s and 1950s probably only highlighted 
the extent to which science normally feeds upon technological demand. 

In education, the question of the relationship of science to technology 
has barely begun to be asked. In a general way, people working at the tech- 
nological, or design, end of the continuum, i.e. , in instructional develop- 
ment, have become aware of the failure of tKe unidirectional model of sci- 
ence and technology. Rarely in their experience have principles from psy- 
chology or the social sciences been applicable in direct form to the problems 
of instruction. At the least, translation has been needed. Sometimes the 
principles themselves have seemed so far from practical concerns that 
developers have felt the need for research deriving from their own ques- 
tions rather than from the theoretical questions of the laboratory ori>asic 
scientist. But mechanisms even for communicating design -oriented ques- 
tions have been lacking, and scientists in education's "base disciplines*' 
have, until quire recently, remained quite aloof from questions of applica- 
tion. 

Occasionally, experimental psycliologists seriously attempting to 
probe the relevance of their own and their colleagues* work to social affairs 
have reached conclusions similar to^bKosc of the designers. An interesting 
case in point is William Estes' (1970) recent survey of several decades of 
research on learning. Estes examined this research for what it could offer 
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as guHehnes in the design of instructional programs for individuals with 
learning disabilities. He explicitly sought empirically validated relation- 
ships between learning, as studied in the experimental laboratory, and indi- 
vidual differences in mental capacity. Estes concluded that although a few 
encouraging developments in both theory and application could be noted, 
"contacts between learning theory and the empirical study of mental develop- 
ment have been sparse and unsystematic" (p. 187). He pointed out, further, 
that where progress had been made, it was generally with respect to the 
severely retarded--!, e. . the most socially conspicuous educational failures. 
This concentration on a specific population meant that little had been learned 
about fostering higher cognitive^processes (see also Estes, in press). 

A few psychologists whose careers have moved relatively freely be- 
tween basic research and applied concerns have addressed the question a£ 
the possibilities for a science of instructional design* Bnmer (1964) has 
pointedvout that a theory of instruction must b^ prescriptive in the sense of 
providing rules for effective ways of bringing about desired learning effects. 
Atkinson, in several papers (see Atkinson, Fletcher, Lindsay, Campbell, 
h Barr, 1973; Groen & Atkinson, 1966), has described procedures for 
"optimizing" learning outcomes by varying parameters of time, item selec- 
tion, and other variables in relation to student characteristics. Suppes and 
his colleagues (Suppes. Jerman. 8c Briar, 1968; Suppes & Momingstar/ 
1972) have devoted intensive study to the detaUs of cognitive performance 
on school tasks, using data from ongoing computer-assisted instruction in 
arithmetic. 

Addressing the question of design in general, Simon (1969), In The 
Sciences of the Artificial , has considered what the requirements for a sci- 
ence of design would be and in what ways such a science would differ from 
the sciences of natural events. Design. SimOn argues, is concerned with 
more than description and explanation. It is concerned with construction 
of artifices (hence "sciences of the artUicial") to meet goals. This i« done, 
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in general terms, by adapting the i>ubstance and organization of the artifact 
to the surroundings in which it operates. Applying Simon's analv^is to edu- 
cation, the artifact is the curriculum or instructional environment, includ- 
ing materials and instructions for vthat do with thum. The "surrounding" 
for an instructional program, the condition that must be understood in order 
to design effective artifacts, is the nature of the learner, both as an indi- 
vidual and as a member of a social group and culture." A science of instruc- 
tional design, following this general conception, would involve the study of 
instructional materials and/or educational environments, the study of the 
learner, and the study of how the learner and the educational artifact inter- 
act, particularly how modifications of the instructional artifacts change 
the nature of this interaction. 

These considerations suggest that a new conception of the relation- 
ship between science and technology, in education as elsewhere, is re- 
quired. This conception must allow for a more symmetrical relationship 
between basic research and social application, thus forcing science into 
active contact with the problems of design. Without such contact, science 
will continue to take its questions as well as its methods from prior theory, 
and its findings will probably continue to be so distant from practical con- 
cerns as to offer little guidance to the designer. To correct this state of 
affairs, to bring science and design- -in the present case, psychological 
science and instructional design- -into fruitful mteriction with one another, 
it is necessary to abandon the unidirectional model of the flow of informa- 
tion between science and technology and substitute an interactive model. 

Figure Z suggests what is meant. The science-to-technology infor- 
mation flow is complemented by a technology-to-science flow,^ communica- 
tion is bidirectional. Science continues to use its carefully controlled 
methodologies and to pursue questions of data or theory that arise in the 
course of such research, but some of its questions are now drawn from 
a'ttempts at social application, from attempts to design products or 
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environments to meet social needs. Similarly, design technology still 
draws its principles from science, and it still depends heavily upon a cycle 
of observation, test, and revision that is needed for honing and sharpening ' 
its products. However, technology now Ukes on a further function. Its 
products and the accumulating data on their functioning can be searched for 
patterns and questions that seem general to a number of products rather 
than specific to any ^e. These are the observations that provide stimulus 
for further scientific work, they are the questions that keep scientific re- 
search in productive touch with problems of design and application. 

The preceding comments on the general nature of curriculum develop- 
ment as design and applied science give the setting within which the des-crip- 
tion and discussion of particular strategies of curriculum development which 
follow should be interpreted. In the following sections, I will describe sev- 
eral curriculum design efforts with which I have been associated that can 
serve as illustrations of how science and curriculum design can interact. 
All are projects undertake n over the past ten years by various groups with- 
in the Learning Research and DeveJoament Center (LRDC)at the University 
of Pittsburgh. Extensive reports on dach of the projects are available and 
will be cited throughout the discussion, but only brief discussions will be 
possible in the course of this paper. 

Throughout the paper, but only occasionally expi^essed, another set 
of assumptions will operate. Curriculum design, like design in all other 
fields, contains elements of both science and art. Instructional design is 
scientific to the extent that it consciously applies validated principles of | 
the natural and social sciences to problems of educational intervention and 
to the extent that it develops and uses systematic methods for testing its 
products and revising them on the basis of performance. It is artistic to 
the extent that inventiveness and intuition are used in initial instructional 
development, and further, to the extent that aesthetic criteria play a role 
in judging the final products. The relationship between disciplined science 
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and artful design should become clearer as some of LRDC's curriculum 
design efforts are described; the possibility and desirabUity of these rela- 
tionships will receive more explicit treatment in a later section. 

Each of the curriculum design examples discussed is drawn from the. 
subject matter of the elementary or preschool and Is thus^cWly allied with 
developmental and child psychology, as well as v.ith learning psychology. 
Further, all havr shared a common commitment to developing programs of 
education that are adaptive to individual needs and abilities of learners. 
They thus have many specific feat^ires in common, such as an emphasis on 
diagnostic description of children's capabUities and some reliance on self 
instructional material. Equally important for the present discussion, each 
curriculum/has been developed with a strong degree of conscious attention 
to deriving general principles from design activity by studying Curriculum _ 
products and their functioning in schools for clues to general principles of 
instruction as well as for effectiveness In meeting an ImmedUte goal. 

Tlie curriculum design projects will be discussed not in their chronp- 
logical older of development, but rather in the context >f a set of considera- 
tions th^t seems to be central both to the development of Instructional prod- 
ucts and to the development of a theory o.' instructional design. ^These con-^ 
siderations were outlined in an earlier paper by Glaser and Resnick (1972X 
They are: (a) analysis of the task properties of the knowledge^ domain or, 
more practically, a description of the state of knowledge to be achl^y-'ed; 
(b) diagnosis of the characteristics of the learners, i.e., descr^flon of 
the learner's initial state: (c) a^^tions that can be taken or conditions that 
' can be implemented to transform the initial state, i. e. , design of the cur- 
riculum or instructional environment;, (d) assessment of specific instruc- 
tional effects, and (e) evaluation of generalised learning outcomes. 



What Is to be Learned: The Analysis of Tasks 

Classically, specification of what is to be learned has been the prime 
requisite for the design of curriculum, [t has often been assumed that ex- 
perts in any domain make the best developers of curricula. It wab this point 
of view that lay, in part, behind the major curriculum reform movements of 
the 19508 and early 1960s. Scientists and other scholars, working at the 
frontiers of their disciplines, began to examine the content of school cur- 
ricula knd found it not to reflect their own understanding of the disciplines. 
The new curricula of the period (PSSC and SMSG, for example) were de- 
signed to present to novices, sometimes even in the elementary school, the 
concepts considered essential b/ experts in the disciplines. 

Instructional design work itself has made it increasing th-.i 
while expert knowledge of the domain to be taught is certainly required for 
curriculum design, the subject-matter expert alone will rarely be able to 
provide a description of his competence in a form that can guide develop- 
ment for n^ovices. Bruner (1964) pointed tliis out when he stress*»d the need 
to transform the expert's version of concepts into forms comprehensible to 
younger or lesa experienced individuals rather than presenting them in their 
original form. More recently, Glaser (1973) has emphasized a distinction 
between knowledge structures as organized for the expert to facilitate use of 
previously learned bodies of knowledge and those organized for the learner 
to facilitate acquisition of competence. , The -.pert's inability to describe 
fully the bases o£ his own performance has also emerged in recent work in 
cognitive psychology, work that attempts to simulate the performance of 
expert performers of complex tasks (e.g., Newell & Simon, 1971). Skilled 
performers of a task cannot always describe well what they know;^ even more 
rarely can they describe the psychological processes called upon when they 
use their knowledge; and they are further still, in most cases, from being 
'ible to describe how they acquired their expertise- -how they changed from 
novices to experts. 
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Task Analyaia in Comntive -DL-velopmental Psychology. 

A distinction between skilled performance and developing competence 
has b;^*tt.e^ed as a general scientific observation by many developmen- 
tal psycho'n«^s. -and study and description of children's competence in 
various kinds of tasks^have been a major focus of cognitive-developmental 
psychology over at least tl^e pas. decade. The tasks studied have frequently 
been drawn f-om Piagefs pioneering work and sometimes from traditional 
adul laboratory learning tasks, now reexamined as developmental phenome- 

Occasionally, developme. :al study of phenomena directly related to 
school task^ such as reading has been undertaken (e.g., Gibson. 1965). 
The results of this large body of research in developmental psychology 
serve to highlight the task facing curriculum designers: The research 
,„akos it ovident that adult concepts and skills are not "born" fuU-blown. 
that children-and presumably, learners of all ages-pass through succes- 
sive stages of understanding and ability en route to what we view as mature 
competence, and that each such stage has a kind of logic to it. -Unskilled- 
performances make sense if understood in their own te rms;. further, they 
aro the route to competence, a route that must be probed and clarified if 
instruction is to optimally match the developing capabilities of learners. 

influential as this research has been in highlighting the general ques- 
tion of how changes in competence occur, the strategies and assun.ptions 
of most research in developmental psychology make it difficult to apply the 
findings directly to instructional problems. • Most developmental research 
is expressly "non-interventionist." The aim is to discover and describe 
..natural" sequences of development. There operates in this work an often 
unspoken but nonetheless powerful driving assumption that the sequences 

discovlred will be universal and environment-independent. "General 
experience" rather than any specific set of environmental interactions is 
though, to produce developmental changes (cf. , Kohlberg, ' 1968; Rohwer. 
,971) Wh I, implied but rarely tested expHcitly in developmental re- 
search is that natural environments are similar in crucial ways, and that 
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they will therefore facilitate development of the »arnt- kind. H.cent cro&s- 
culturalwork (see Cole, Gay, r,lick, ^ Sharp, 1971. Goodnow, in prcbS, 
Pick, Note 1) is beginning lo ..ubstantiate the long-ajgued i,uyg<'stion that 
environments may differ quite radically in their demands and that adult cog- 
nitive competence and style vary accurdmyly. luit developmental pbvcholo- 
gy as a whole still lacks a theory and even a i;oo(P.->et of dcscriptorb of the 
envirunirent. vThus, u lb diffn uh to cull iroii * de velopmental theory any 
but the most general u-iidehn^^ fur design of i-nvi. unrnents ^specifically in- 
tended to bring about clian>»fs in competence. 

j^jL^iP"*'^^ Task Analysis and Learning Hierarchies 

What is needed for purposes of instructional design is a description 
of desired educational outcomes in a form that specifies the psychological 
processes called upon ui skilled i^crforniance and also suggests less sophis- 
ticated organizations of skill and knowledge that are capable of leading to 
or producinkj skilled performance, in other words, we need analyses of 
learning goals that ^pecifv both the nature of competent task performance 
and simpler tasks that will facilitate learning. These "facilitativc" tasks, 
organized and sequenced to niaxinu^u the match between current abilities 
and new demands, will constitute a curriculum (ci. Hunt, 1961, for a dis- 
cussion of this matching proce.s.s from a different tlieoretical viewpoint). 

Over several years of v^ork in designing an early learning program 
for preschool and prim.ar;y grade children {The Primary Education Project,^ 
see Re snick, Wang. ^ Hosm^r, m press), wo have developed a method of 
task analysis explicitly designed to meet these objectives. The strategy 
yields hierarchies of tasks simil.ir to those proposed by Gagne (1962, 
1968), but with more specification of the actual processei, involved in per- 
formance. A detailed description of the method as applied to an introduc- 
tory mathematics curriculum appears in Resnick, Wang, and Kaplan (1973^). 
Figure 3 illustrates the process. Each box in the figure describes the 
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situation as presented to the child atid the child^s expected response. Thus 
the top box (la) should be read, " Given a set a moveable objects, the child 
can count objects, moving them out of the set as he counts." This provides 
verbal specification of the task to be analyzed. The first step in perform- 
ing an analysis is to describe in as much detail as possible the actual steps 
involved in skilled performance - ^i. e. , the "components" of the task. The 
results of this analysis are shown in level II of the figure. Once the com- 
ponents are identified, each is considered separately, and abilities that are 
prerequisite to or facilitative of learning the components are identified. 
For example, the tasks described in IVa and IVb facilitate learning lUa, 
which in turn facilitates learning Ila and lib. Analysis of this kind can 
result in charts showing several levels of tasks with complex interrelation- 
ships among them (shown in the figure by connecting lines). Frequently, 
when a set of tasks from a single curriculum domain are being analyzed, 
one task in the set to be analyzed appears as a prerequisite to another in 
the set. These relationships provide the basis for sequencing the separate 
tasks of a curriculum in a way that will optimize transfer and maximize the 
likelihood of success in learn'ng. Frequently, too, the same abilities ap- 
pear *as prerequisite to several different tasks. In this case a generalized 
ability has been identified, one to which special diagnostic or instructional 
attention should probably be devoted in order to optimize instructional effec- 
tiveness and efficiency for learners of varying characteristics. 

Although the example given here is for a very simple task, counting 
a set of moveable objects, the strategy of analysis is a general one and can 
be applied to tasks in various domains. The level of specificity- -that is, 
the fineness of detail with which given tasks are analyzed--depends upon 
the task and the population for whom instruction is being designed. As a 
g"neral rule, no further prerequisites would be specified when a level of 
behavior is reached that can be assumed in most of the student population 
in question. Further, the initial analysis should describe task components 

14 



ERLC 



at a level of detail conbonant with what one expects is already a reasonably 
well organized behavior m the learner's repertoire. Even in a curriculum 
for kindergarten children, as in the present example, much is left unspeci- 
fied. For example, it is assumed (Box Ua) that the child will be able to 
recognize a grouping of separate elements. Similarly, it is assumed that 
the motor acts of moving an object or saying a word are already organized 
and need not be further analyzed. 

We have used these strategies of task analysis in developing curricu- 
la for a variety of preschool and primary grade programs {see Beck ti 
Mitroff. 1972. Resnick et al., in press;; Undvall, Note Zh In each case, 
careful attention has been devoted to ^lidation of the analyses either 
through special validation studies (see Resnick, 1973, for a general dis- 
cussion of validation strategies) or through careful study of the curricxUum 
itself (see Resnick et al. , 1973). These studies not only serve to validate 
proposed curriculum sequences, but also yield basic information on natural 
sequences of development (e.g., Wang, 1973; Wang, Resnick, & Boozer,'' 
1972) and on the nature of transfer relationships among hie r a ire hie ally 
related tasks (e.g., Caruso & Resnick, 1972; Resnick, Siegel, «c Kresh, 
1971). Thus, they exemplify a form of interactive research and develop- 
ment, cases in which efforts to develop optimal curricula have generated 
research on questions of basic scientific interest. 

Information Processing Analyses 

The analyses juot described can be appropriately characterized as 
informal "process models" ^f performance at different stages of exper- 
tise. As process models, they specify hypothesized components and tem- 
poral organization of performance and attempt to identify the underlying 
memory, perceptual, linguistic, and other processes tJiat are embedded 
in the more complex tasks. More formal methods of task analysis, drawn 
largely from the work of current information processing psychology^ also 
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deserve consideration as possible bases for future instructional work. 
Formalized task analyses require both more completely specified per- 
formance iT'odels than the ones we have used in curriculum development 
up to now and more precise empirical validation. The most demanding 
current criterion for a complete performance model^is that it be written 
as a computer program that is capable of performing ^he task. A pro- 
gram of this kind encourages precision in modeling {computers take their 
instructions quite literally) but can be written so as to engage in quite 
sophisticated "heuristic" behaviors (see Newell & Simon, 1971). 

Technologies for computer modeling of this kind exist. It would be 

a tedious but not highly inventive task to turn each of the many informal 

\ 

procesn modclft we have written into programs that assured us of the com- 
pleteness-'in this technical sense- -of our analysis. With that task accom- 
plished, we would still need to test the programs as models of human per- 
formance. Our work on validation of learning hierarchies has provided an 
initial test of these models. The models are accurate enough to predict 
well the transfer of relations holding between tasks and the general order 
of acquisition. However, a variety of specific performances might pro- 
duce the same general patterns of sequence and transfer; more rigorous 
methods of validation are required in order to lend credence to the details 
of any given process -v^odel. Methods for guch validation include detailed 
analysis of individual verbal and behavioral protocols, study of differences 
in reaction times for slight variants of a task, analysis of frequency and 
type of errors, and so forth- -in short, virtually the entire range of meth- 
ods now in use in the field of cognitive psychology. 

Clearly the use of formalized models is expensive, and consideration 
must be given to when the added cost of formal specification and empirical 
study is likely to add enough instructional power to make the effort worth- 
while. In subject-matter domains where much is known--where successful 
examples of instruction abound, where the subject matter has an inner logic 
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that lends strong face validity to hypothesized analy8CB--the added effort 
of formalized analysis may have relatively small direct payolf for instruc- 
tion. Where Httlc is known, where even strong hypotheses concerning 
actual process are difficult to find, much may be gained through the disci- 
pline of formal analysis. Many unlikely or less than optimal p^ths for 
instruction may be eliminated, and strategies likely to be effective may 
be suggested. 

A contrast between analysis of mathematics and reading tasks at 
different stages may help to make this point. In a general sense, mathe- 
matics - -especially early mathematics of the kind we have studied and ana- 
lyze^!- -presents relatively few ambiguities. The subject matter Is a closed 
logical system, within which informal analyses are quite likely to reflect 
skilled performance reasonably faithfully and thus to guide Instructional 
design effectively. The same is largely true of early reading, where 
letter -sound correspondences are relaiiveiy easily specified and ordered. 
By contrast, it is very difficult to develop process descriptions for lan- 
guage comprehension and for advanced mathematical thinking. Many com- 
peting analyses are possible. Especially In language, the subject-matter 
domain is "open" and highly dependent on context, on past experience, and 
on details of the task as presented. It is by now quite widely agreed (see 
Miller, in press) that comprehension Is not a simple linear process, thtt 
it involves use of redundancy in the structure of the language and a search 
for meaning --in short, it is a very complex cognitive process. For this 
kind of subject matter, formal process analyses and empirical tests are 
likely to have especially high payoff. At present, for example, there 
exists no generally accepted model of how a prose passage Is compre- 
hended when read, of the processes Involved In taking and Interprettug 
information from a text. In this situation, much may be gained through 
the disciplining effort of formalized task analysis and related empirical 
research loading to the development of psychological models of compre- 
hcnsion processes. t^^J 



17 



ERLC 



One example of a case where a formal analysis of an academic task 
has led to some unusual and potentially highly productive instructional 
design comes from spelling. An original model for how children spell was 
written by Simon and Simon (19?3), This was a formal model in the sense 
that it (a) took Into account existing data on the structure of the sound- 
symbol correspondences of the language and the typical errors made by 
children in spelling, (b) was written as a computer program, and (c) was 
validated, in part, on the basis of the ability of the program to match 
actual performance both in terms of correct spellings generated and in 
terms of the kinds of errors typically produced. Spelling programs that 
embody some of the processes identified in the Simon and Simon analysis 
have been written under the direction of Karen Block at the Learning 
Research and Development Center. These spelU^ig lessons, which are 
computer-based and utilize the special rejponsive characteristics of the 
computer, include practice In generating alternative spellings for given 
sounds, in sorting spellings according to patterns used to represent the 
same sound, and in choosing among alternative spellings. This work vir- 
tually embodies the interactive mode of curriculum design. The Initial 
analysis of spelling was motivated by an applied instructional need. The 
resulting model provided the basis for lesson design, which in turn will 
provide one of the pieces of converging evidence needed to validate the 
model itself. If children traineJ in the processes identified in the model 
improve in spelling and continue to match the program in terms of quality 
as well as correctness of output over successive levels of competence, 
validation of the model will have been provided in the course of instruc* 
tion itself. 

Formal and empirical process analyses are likely to yield much 
instructional power when the aim is to identify and find ways of instructing 
what we call "generative skills"- basic abilities that are used in a variety 
of learning and other performance situations. Current cognitive psycho- 
logical theory is pointing with increasing consensus to the existence of 
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certain basic processes —for example, memory, perception, infet-ence 
strategies- -that are quite general in human functioning and thus would 
produce generalized learning abilities were we to find ways of enhancing 
them. These abilities are embedded in academic Usks and in on-the-job 
performance tasks of various kinds. Some of these abilities appear to be 
tapped indirectly by the kinds of tasks that are found in most widely used 
intelligence and aptitude tests. But the abilities are deeply embedded: 
they are not visible in cursory forms of taok analysis ao, for example, in 
job analysis techniques, and they require complex st^idy to be detected and 
understood. It seems likely that fornnal task analysis procedures wiU prove 
to be a very effective means of identifying generative skills and suggesting 
methods for their instruction. The power of such analyses in the instruc- 
tional design domain remains only a promise at this time, but it is a 
promise worth serious investigation. (For an early example of some 
possibilities along these lines, see Resnick & Glaser. in press; for a 
more extended discussion of the general role of information processing 
task analysis in instructional design, see Resnick. in press. ) 

Diagnosis of Learner Characteristics 

Closely allied to the problem of describing what is to be learned is 
the problem of describing the capabilities of the learner as he enters the 
learning environment. This step is crucial if curricula are to be adaptive 
to individual differences. It is necessary to know what the learner is 
like--what he can do that is directly relevant to the subject matter to be 
learned, what he can do that constitutes a more generalized learning-to- 
learn skill, what motivates him, and so on. With '-is knowledge in hand, 
it becomes possible to design curricula that adapt to these characteristics. 
The description of these characteristics must be in terms relevant to edu- 
cational decision making. For this reason. t>T)ological descriptions gen- 
erally are not useful, as compared with descriptions of specific competen- 
cies and behaviors that can be observed under specifiable conditions. 
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Describing individuals as highly intelligent or not so intelligent gives httle 
guidance in what kinds of instructional treatments are most likely to be 
effective. The ineffectiveness of such approaches is testified to by the 
bulk of the literature on aptitude t*eatment interactions- -years of research 
that have yielded virtually no useable information on how to adapt ij*struc- 
tional treatments to individual differences (Bracht, 1970; Cronbach & Snow, 
1969). Thus traditional approaches to testing and differential psychology, 
developed and validated for purposes of selection and prediction rather than 
adaptive instruction, are of relatively little value in their present form. 
New ways of conceptualizing the nature of individual differences are re- 
quired and, along with them, new ways of assessing these differences 
(Glaser & Nitko, 1971; also Glaser, 1972). 

Describing the learner is not something that is done at the outset of 
instruction and then left alone until final assessment of the outcome of in- 
struction. Rather, it must be a i.iore or less continuous process. As he 
proceeds through a curriculum, the learner's capabilities change. They 
change m tern.s of both specific knowledge acquired and generalized abili- 
ties available for further learning. These changes can result either from 
general experience during the period coterminous with instruction or from 
engaging in the instruction itself. Thus, to maintain an effective curricu- 
lum over an extended period of time, it is necessary to address the prob- 
lem of describing changes in the learner's capabilities as instruction pro- 
gresses and to do this in a form that is useful in making decisions as to the 
course of succeeding instruction. 

The discussion in the previous section on analysis of the subject mat- 
ter has already alluded to some of the ways m which thus problem can be 
attacked. Hierarchies of learning tasks, generated through formal or 
informal task analysis, provide a way of desciibing the learner's^capabili- 
ties in terms of the tasks and underlying abilltieb that make up the curricu- 
lum itself. Describing which tasks an individual can already perform 
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within a hierarchy and which are yet to be learned tells the instructor (or 
the learner himself) which tasks he is likely to^make progress on and \ 
which he is likely to find too difficult. Thus the learner^s capabilities 
can be described in terms of the curriculum content itself and its under- 
lying processes. Further, they can be described as a changing function of 
his progress, or lack of it, through the curriculum. The hierarchy of 
tasks thus serves as a map that describes both present location and pos- 
sible directions of movement of any learner currently under consideration^ 

The Role of Criterion- Referenced Testing 

Implementing this mapping functipn of curriculum hierarchies re- 
quires a form of testing that is explicitly designed for ongoing instructional 
decision making- -i. e. , that will help in placing individual children in the 
curriculum structure and provide diagnostic information for use in making 
specific instructional assignments. Considerable effort was devoted to 
developing such a testing strategy in LRDC's first major curriculum de- 
velopment undertaking. Individually Prescribed Instruction (IPX). A paper 
by Glaser (1970) outlined the major evaluation requirements for a system 
of education that adapted instruction to individual x*equirement8 : 

L Outcomes ... of learning are specified in terms of . 
observable learner performance and the conditions 
under which this perfornuince is to be exercised. 
\ 

2. Detailed diagnosis is niade of the initial state of a 
learner entering a particular instructional situation. 

3. ]^ Educational alternatives are provided which are adap- 

tive to the classifications resulting from the initial 
student ability profiles. 

4. As the student learns, his performance is monitored 
and continuously a^ssessed at longer or shorter inter- 
valb appropriate to what is being taught. 

5. Instruction and learning proceed in interrelated fashion, 
tracking the performance and selections of the student. 

'21- 



ERLC 



i 

i 

\ 

6. The instructional system collects information m order 
to improve itself, and inherent in the systerri*s design 
is its capability for doing this. (pp. 29-30) 

These requirements were met m IPl by elaboration and application 
of the notion of criterion-referenced testing (Glaser, 1963). Such tests 
are equivalent to "work samples" as a means of assessing job perform- 
ance. Criterion-referenced tests were used to assess entering level in 
the curriculum sequence (placement tests), immediate mastery of indi- 
vidual skills as they were taught (curriculum -embedded tests), and mas- 
tery of a unit of related objectives (pre - and posttests). With younger 
children in the Primary Education Project (PEP) and subsequent adaptive 
curricula the tests were administered orally on a one-to-one basis and 
frequently involved the use of manipulative materials appropriate to the 
task and age level. These tests were used by teachers as part of the regu- 
lar instructional process. The tests, along with observation of the chil- 
dren's activities as they vvorkod on instructional materials, guided the 
teachers both in placing children m the curriculum structure and in adapt- 
ing the instruction to individual needs- 
Use of criterion- referenced tests m the manner described for indi- 
vidualized instruction raised a scries of questions which m subsequent 
years commanded the attention of researchers within LRDC and elsewhere. 
One set of questions has concerned the strategy for jjecifying objectives 
and creating pools of test items that would adequately sample the specified 
domain. Initial test development efforts were informAl and iterative: An 
objective was described verbally, test constru'ctors wrote items that ap 
pea red to match the statement of the objective, the items were reviewed 
by the curriculum designer, and either items or stated objectives or both 
were revised until a consensus was reached that the tVBt reflected the cur- 
riculum developer's intent. There was no formal procedure for specifying 
the objective to be tested or for sampling among test items for inclusion 
in the tests. Further, standards for passing and failing, nvistery and 
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nonmastery, were informally and largely arbitrarily set in terms of ^ 
passinj;^ or failing some fixed, number of items or* the test. A recent 
paper by Nitko (1974) discusses m some aetail the questions posed for 
testing theory and practice by the increasing use of criterion-referenced 
tests. 

Some Examples of Nt'W Approaches to Testing for Instructional Decision 
Making 

In the case of test theory, as in the case of task analysis, serious 
work on theoretical questions has been prompted by the demands and the 
:)uccesses of instructional development. Two ca jes of interwoven develop- 
ment and research activity in testing illustrate this point. The first con- 
--c^ns the use of the computer as an aid in "tailored" or "adaptive" testing* 
It 13 possible, using a computer, to tailor testing to the individual's per- 
formance by limiting the number of items given to those actually needed 
to reach a decision and by branching to new classes of items for further 
diagnosis. To accomplish this, systematic rules are required both for 
genf:rating items in a class and for making pass-fail decisions. The first 
requirement was mot by an adaptation of the strategy of "item forms" 
(see Hively, Patterson, &t Page, 1968) so that the computer could generate 
a large number of specific items based on a limited set of stored parameters 
and generation rules. Two diffe rent^ kinds of decision rules, one based on 
traditional statistical theory and the other on Bayesian theory, have been 
investigated. Both permit eva* lation of successive "pass" and "fall" cases 
as they cumulate, and both allow testing to stop as soon as a reasonably 
certain decision is reached. This is much the way a skilled diagnostician 
proceeds: If the individual being tested passes several items in sequence^ 
a positive or "mastery" decision can be made; if he falls severrl In se- 
quence, a negative or "nonmastery" decision can be made; If the Individual 
vacillates between passing and failing items, a longer period of testing Is 
needed to reach a decision. Further detail on these theoretical Investigations 
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and their applications to t»\ilored testing can be found in papers by Fergu- 
son (I969)and Hsu. Carlson, and Pingel (1974). 

The second case of interest here grows out of the successful develop- 
ment of diagnostically oriented tests of visual and auditory skills by Jerome 
Rosner (1971, Rosner ti Simon, 1971). The tests Hosner developed corre- 
late positively and strongly with existing tests (m the case of visual skills) 
and also predict performance on related academic tasks such as beginning 
reading (iii the case of auditory skills). They are distinguished from past 
tests of these skills by the fact that they are tied directly to curricula de- 
signed to teach the skills tested. Typically, tests used m predicting aca- 
demic performance lose their predictive validity when the tasks appearing 
in the tests aie directly taught. Studies conducted to date suggest that this 
is not the case for Hosner* s tests --that instruction oriented to improving 
test performance can be successful and that the improved performance 
will be reflected on other tasks not directly taught. This kind of finding, 
if replicated, validates the procedure of using critervon-referenced tests 
as cargets for instruction. It requires, however, thai the tests be vali- 
dated for such use- -I.e., that their ability to retain predictive validity 
when directly taught is established. A recent working paper by Nitko 
(Note 3) discusses the kind of validation studies necessary for criterion- 
referenced tests that are to be tised as an integral part of instruction. 
Such validation studies represent a potential new methodolpgy m test con- 
struction. 

Design of Instrugrfional Materials and Environments 



A major and in>jiortant aspect of curriculum design work has been 
completed v/hen. task analysis and diagnostic instruments are available. 
The task analysis translates general statements about cur*riculum content 
into specific concepts and skills to be taught and, in many cases, klao pro- 
vides a sequential structure for the content. Criterion-referenced 

t ^ m 
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diagnostic tests, m addition to their use in instructional decision making, 
serve as working definitions of the specific content. 

Yet despite the structure imposed by task analysis and criterion- 
referenced tests, there is a large, and largely intuitive, leap to be made 
from these specifications to an interesting and effective instructional pro- 
gram. Examination of "systems" approaches to curriculum development 
will reveal that there is no good set of prescriptions a*/ailable for design- 
ing instructional materials. Instead, there is heavy dependence on a cycle 
of iterative development and testing. This process serves to validate the 
effectiveness of intuitive translations of curriculum specifications into 
instructional procedures; it cannot, however, substitute for artful design 
work at the outset, for tests will only reveal successes and failures, not 
prescribe alternative strategies. The actual design of instYuctional materi- 
als or instructional environments, then, is a matter of art as well at ici- 
\ 

ence and requires a special facility for combining intuitive and scientific 
knowledge. 

Design of Adaptive Learning Environments 

The need for an effective combination of art and science bccom'ft*-^^ 
especially apparent when one is concerned with designing entire environ- 
ment? for learning rather than the sej^arate components or instructional 
mp.terials. A "curriculum" is often thought to consist of the "leisons" 
that students will encounter --the "things" of instruction. Only rarely in 
discussions of curriculrjn does one find much explicit treatment of the 
role of the total environment in promoting (or hindering) learning. Yet 
the social context wUhin which instructional materials arc used can 
enhanc#» or diminish their effectivenes3 to an extraordinary degree and, 
conversely, Instructional materials can encourage or discourage certain 
kinds of social expectations and behaviors that are essential to learning. 
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The history of LRDC's work in designing adaptive educational pro- 



grams illustrates the importance of the general environment as an instruc- 
tional variable and the ways m which instructional materials and social 



attempt to apply general principles of individualised ins ruction to an 
entire eleirentary school. It was not a lesson design project, but rather 
a school design project, with the necessity for curriculum development 
folic ing from a conception of how a school adaptive lo inciividual differ- 
ences ought to run. 

The goal of IPX was to develop a form of instruction that would em- 
body principles of individualization in a form that was dissemmablc to and 
useable in a broad array of public schools. Individualization had a long 
history in educational theory and m relativelv isolated instructional prac- 
tice, but in the early 1960s it had not been shown to be a principle that 
could work in practice outside of a few specially staffed demonstration 
schools. To translate individualization into a widely useable concept, it 
seemed necessary to provide a manageable mean^ for teachers to diagnose 
children's strengths and needs m any given learning domain and then to 
provide instruction suitable to these diagnoses. 

Adapting Existing Materials to an Individualized Environmen t 

Initial efforts of the IPX project were directed at identifying and 
organizing existing instructional materials for individualized use rather 
than designing new lessons or related material. Thus, in keeping with 
the principles discussed earlier, worK proceeded on specification and 
sequencing of objectives and on design of tests for monitoring student 
progress. The intent was to use the tests diagnostically and tlien make 
assignments in a variety of texts and workbooks to match individual needs. 
These early efforts made clear the extent to which instructional materials 
and instructional environments were interdependent. The existing textbooks 



environment^ for learning interact with one another. IPX began as an 
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had not bee:. -Arritten for an individualized school, and it proved more diffi- 
cult than expected to key ex. tin^ materials to IPI sequeoces of objectives 
and tests. It was not easy to find in the texts lessons tliat fully taught new 
concepts, since it was generally expected that these would be presented by 
the teacher in group instruction and that the exercises engaged in by chil- 
dren would serve largely as practice. Further, close analysis revealed 
that most textbook lessons confounded the teaching of several concepts or 
skills. The lessons were often designed simply to expose children to con- 
cepts rather than to assure mastery of those concepts. These features o£ 
the textbook materiaU discouraged independent work by children and thus 
miplicitly encouraged the continuation of teacher-oriented Instructional 
practice. In a public school, even one that enjoyed a greater than usual 
number of paraprofessional aides, this inevitably meant group instruction 
and expository rather than participatory instructional styles--in short, a 
less than optimally adaptive environment. 

Dcsignin)^ Instructional Materials to Shape the Environment 

These observations, based on trials in an entire elementary school, 
led to increased attention to the development of instructional materials 
themselves. The IPI and PEP curricula, as they are publicly known and 
used today, are the results of this "materials development" phase of 
LRDC's work. These curricula continue to embody a general system for 
managing structured individualized teaching. Heavy emphasis is placed 
on the specification of learning objectives in observable form and on test- 
ing that allows placement of children in the various curriculux^ sequences 
according to their actual level of mastery at a particular time. Emphasis 
IS also placed on keying insti actional materials to tests so that assign- 
ments can be matched to children*s performance on these testa. Floally, 
tests and instructional materials are arranged in such a way that children 
can access and use them on their own or with the help of paraprofessional 
aides, thus permitting them to work independently of the teacher's moment* 




to-moment direction and with a considerable degree of self-management. 
The result in practice has been a substantial difference in the rates at 
which children progress through the curriculum (Glaser, 1968, Resnick 
et al. t 1973). The program has also resulted in some substantial changes 
in social interaction patterns in the classroom. In various studies teachers 
have been observed to engage in more instructional (as opposed to manage- 
ment or discipline-oriented) interactions with children and more of these 
have involved individual children rather than groups. There are more 
student-initiated communications, and there is generally less teacher* 
oriented behavior on the part of the children. Thus, this instructional 
design effort has demonstrated that substantial amounts of individualiza- 
tion, at least with respect to rate of learning, are possible in the public 
school classroom, given appropriate design and organization of the instruc- 
tional materials. It has also made it clear that such individualization, if 
seriously pursued, has potentially profound implications for the social 
organization of the classroom. 

Desij^ns for Organizing and Running an Adaptive Classroom 

LRDC*8 curriculum development efforts led to the realization that 
although individualized instructional materials encouraged adaptation to 
individtial differcncefi. the^ could not, in and of themselves, ensure it. 
Indeed, quite new forms of classroom organization and teacher behavior 
were required if the curricula were to function optimally. With 25 or 90 
children working on almost as many different activities, the traditional 
model of the teacher demonstrating or explaining to groups of children 
could no* work. Some teachers, given the instructional materials and the 
help of an aide for administering or scoring the various tests that iormcd 
part of the program, invented for themselves a way of managing this com- 
plex social environment. Others foundered. 
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Observation of and consultation with teachers who were successful 
m Uhing the program eventually led to the development of a model for 
classroom management and organization. The model specified open dis- 
play of instructional materials and coding system that would allow even 
a very young child to interpret instructions on a "ticket" that specified 
•*hich activities he should engage m. It also specified the various teach- 
ing functions that needed to be cairied out in the individualized classroom: 
testing, tutoring individuals, instructing small groups, and "traveling" 
among the children as they worked to help, to encourage, and to observe. 
Traveling was the function most foreign to teachers used to traditional 
classroom organizations, and new teachers needed considerable help in 
learning the most effective ways of behaving as they circulated in the class- 
room. Teacher training strategies had to be developed for this and related 
forms of behavior (see Leinhardt, 1973). As the program was disseminated, 
these new modes of interaction bftcame incorporated as a vital part of the 
definiticn of the program, thus broadening the "curriculum" to include 
explicitly the social environment created by the teacher (see Resnick ct al. , 
m press). This broadened definition of curriculum, which includes goals 
of self-direction and general learning (inquiry) skills and incorporates 
strategies for managing a classroom in which complex combinations of 
assigned and child>selected activities occur simultaneously, is also em- 
bodied in LRDC's Individualized Science program (see Klopfer & Cham- 
pagne, 1974). 

Adapting Curricula to Aptitudes 

Observation of the iPI programs in use and study of the cumulating 
test data on individual children made it increasingly evident that for some 
individuals, assumptions the curriculum made concerning their abilities 
upor. entering the program were incorrect. In common with other primary 
grade programs, IPI assumed a wide range of specific abilities and gen- 
eral "aptitudes" upon entry to school. Ability to focus attention in a 



directed way, to analyze visual and auditory inputs, to hold material in 
memory in the face of competing s timula tion - -all of these are abilities 
oalled upon in a large number of different school tasks. They are thus 
examples of the learning-to-learn abilities Q.scussed earlier in this paper. 
They are aptitudes that mast be explicitly taken into account in a fully adap- 

tive curriculum that aims to assure mastery for all children (cf. Glaser, 

1972). 

1\vo kinds of adaptation to individual aptitudes must be given careful 
consideration in instructional design. One is the possibility of teaching 
aptitudes directly so as to insure that all individuals entering a given in- 
structional program have at their command the basic process abilities 
assumed by that program. The second is to design alternative instruc- 
tional treatments to match orcater to the differences in entering aptitudes 
within any given population. We have explored both of these approaches. 

The notion that aptitudes may be teachable is relatively new both in 
psychological theory and m educational thought (see Glaser ^ Pesnivk, 
1972. for a review of literature in the area). One of the few aptitude 
domains that has been studied m this way in the past is that of perceptual- 
motor abilities. The relationship of visual and auditory abilities to early 
school performance has long been recognized, and over the years consider- 
able effort has been devoted to training children recognized as having per» 
ceptual-motor dysfunctions. Rosner's work in this area, mentioned earlier, 
extended this concern to preschool and kindergarten children and was aimed 
at £reEarijifi_them for the academic demands of reading and mathematics 
instruction rather than at offering remedial help once a cycle of failure had 
manifested itself. In addition. Rosner engaged in both systematic analysis 
and detailed, iterative testing of instructional materials in order to come 
up with a set of activities that would tram the underlying processes of 
visual and auditory analysis. The result, developed over a period of about 
five years, is a set of teachinv? programs for visual and auditory analysis 
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skills (Rosner, 1973) that are capable of raising scores on standardized 
"aptitude tests" and also show transfer to learning early reading and 
mathematics (Rosner, 197E). *^ 

While programs that teach basic Jfptitudes \re extremely powerful 
because o£ their generative relationship to other learning, the*' are not 
always available or useable. When aptitudes cannot be taught, adaptive 
education requires that instruction in other programs be matched to the 
learner's abilities. Our work on modification of the initial IPI reading 
program illustrates this form of adaptation. The initial program was 
based on a commercial programmed reading series to which we had added 
instructional tapes for teaching new sounds and for a variety of additional 
story reading activities. The program was a phonetically oriented one and 
it assumed, after the introductory unit, that children taught a new letter- 
sound relationship would be able to apply it to the "sounding out" of new 
words. Yet a crucial part of the sounding -out process was never really 
taught. That part is known as "blending, " The program we used, like 
most available beginning reading programs, told children to blend ("slide 
the sounds together",' "say it faster") but never taught them how . Some 
children, presumably those with better developed auditory skills, invented 
a procedure.^ others did not and as a result continued to have difficulty with 
later units in the reading program, 

To meet the needs of these children for more explicit instruction, an 
introductory component was added to the reading program on a trial basib. 
This component taught children explicit strategies for blending sounds. 
The results were excellent: There was close to a 100% success rate in 
learning the blending process, and performance on standardized as well as 
curriculum -specific reading tests improved. This blending strategy is now 
being incorporated into a beginning reading syLtem that combines a variety 
to types of instruction into a highly adaptive program (see Beck k Mitroff, 
1972). This form of adaptation to aptitudes offers special support to the 
learner in an area in which 1.^ is weak in the context of a regular subject- 
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matter program. It is a strategy that needs substantial further investiga- 
tion. One potential effect^ suggested by occasional studies in the past 
(e.g., Meuris, 1970) but iie\er seriously investigated in the context of 
curriculum development, is mat the aptitude itself may be increased by 
subject-matter instruction of certain kinds. 

The examples discussed here are representative of design strategies 
that incorporate long-term study of curriculum effects as well as short- 
term trials of leasons. New approaches to instruction rather than "patching*' 
of existing programs are often suggested by such study. These long-term 
studies are costly ^n their face, but may be inexpensive when considered in 
terms of ultimate effectiveness. The general strategy is to design and 
implement a curriculum relatively quickly, building on the best available 
knowledge combined with a good measure of artistry and invention. Once 
a curriculum is operating, the needs of the current population of students 
are being met while at the same tire a natural laboratory is formed in 
which research can proceed. In this way the cost of successive program 
development is spread over several years of actual use. Meanwhile, re- 
srearchand observation focused on actual instructional effects can proceed, 
furnishing another example of an interactive research and development 
mode. 

Assessment and Evaluation 

Earlier sections of this paper have already alluded to the kinds of 
strategies we have found most useful m assessment of specific instructional 
effects. The use of criterion-rc ^;rencefl tests as a part of the curriculum 
has made it easy to follow the progress of children and thus to test the effec- 
tiveness of our program in teaching its defined content. Papers by Glaser 
(1967) and Resnick et al. (1973) describe the way in which this monitoring 
process can be used in study and revision of curricula and give examples 
of the kind of data collected. 
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Other studies (e.g., Pesnick & Wang, 1974) have shown a substantial 
relationship between position within the curricultim hierarchy stnd perform- 
ance on standardized tests, thus offering an addition?! validation of criterion- 
_referenced tests and the general program strategy. 

Not all objectives of instruction lend themselves easily to frequent 
criterion-referenced testing. This seems to be the case for domains such 
as reading comprehension, scientific inquiry, problem solving, and other 
abilities that are complex and heuristic in character* It is likely that such 
abilities develop over relatively long periods of time and that the cumula* 
tive effect of the total environment- -instructional materials, questions 
raised by peers and teachers, the opportunity to risk new tries — rather 
than any specific set of lessons produces the effect. This raises a n3w 
set of tasks for those concerned with assessment of instructional effects* 
If all assessment is left for the long term there is no way of determining 
which acpscts cf zt. ir.ctruttional program produced the effects obctJrvsd 
nor any way of adapting instructional strategies ap wurk proceeds. New 
methods of assessment, probably involving observation of the educational 
process rather than tests, will need to be developed. Again, though costly, 
such development work is likely to yitld rich dividends in educational prac- 
tice and theory by providing more systennatic and useable knowledge about 
how generative learning skills develop and how environments for teaching 
them can be designed. 

Evaluation in the Field i 

Ultinrvately, the value of any curriculum depends upon how it works in 
the field awaj^rom the watchful and sympathetic mii.I.itrations of its de- 
velopers. Classically, the pattern for field evaluations has been to con- 
trast settingb using some program with settings not using it. Such evalua> 
tions have rarely yielded clear results, nor have they ever been very useful 
in helping to determine which elements of a program were crucial to 
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m»int»in »nd which could be dropped or modified. One reason for the lack 
of clear contrast effects has been suggested earlier in this paper: No two 
teachers Implement a program in quite the same way. In order to inter- 
pret field test data, it is important to know how the progra.m has been im* 
plemented in each site studied and to include implementation data in inter- 
preting outcome data. ^ 

Cooley (1971) has oujblined a general strategy for using implementa- 
tion data in field evaluation of curriculum innovations and the method has 
been applied both to LKDC's programs and to a portion of the Follow 
Through national data (see Leinhardt, 1974a, 1974b). This methodology, 
along with related strategies that use implementation measures as part of 
an evaluation study, is of substantial interest for its contribution to inter' 
active research and development- in curriculum. Not only does it yield 
more Inte rpretable information on outcomes in the field, it also serves to 
te«t--or generate--hypotheses concerning which instructional effects are 
the most powerful. It server, in other words, to abstract from specific 
programs the features that contribute most to learning outcomes. This 
lnforn«ition in turn offers guidance In new curriculum development that 
may incorporate important common features Into programs quite different 
in other respects. 

Conclusion 

This paper began with a discussion of curriculum design as an applied 
science. The succeeding sections focused largely on describing the present 
and potential character of an interactive mode of instructional research and 
development that might yield both curriculum products and scientific knowl- 
edge concerning learning and instruction. It remains to make more explicit 
the assumptions noted earlier concerning the relationships betweer art and 
science. By way of conclusion, I turn now to a consideration of the place of 
art in the science of curriculum design. 
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Various fields of design have differed in the relative ernphasis placed 
on scientific and aesthetic principles and criteria in their work. In some 
fields of design- -for example, architecture --the artis'tic function is explic- 
itly recognized and accorded status. The designer's function is not simply 
to apply knowledge produced elsewhere in a mechanical and deterministic 
fashion, but to use this information to create a product that is. aesthetically 
pleasing as well as functional and economical* Architects, however, can- 
not function without knowledge of a significant amount of basic physical sci- 
ence as well as knowledge of available materials and their properties. Thu 
the profession must also qualify as an applied science. 

In fields where aesthetic criteria for the finished products are less 
pronounced, artistry in creation tends to be played down, and systematic 
procedures for developing and evaluating products are stressed. This 
is the case in most fields of engineering, which stress their technological 
(systematic) character and the scientific principles underlying their work. 
Yet engineering frequently involves artistic processes as well. Heuriiticc 
and intuition play a role in the design process- -later to be checked and vali- 
dated systematically- -and aesthetic criteria are frequently applied in the 
form of elegance, gracefulness, economy, and related features of the 
product. 

Curriculum, as a rank newcomer among fields of design, has not yet 
developed a firm inuige or stance on the science vs. art dimt^nsion. This 
is perhaps fortunate since it allows us, in conceiving of a discipline of cur- 
riculum design, to view it as both a science and an art. Curriculum design 
like other fields of design, requires a delicate balance between artfulneso 
and scientific rigor. Without art, the curricula and instructional programs 
produced are likely to be pedestrian- -uninteresting to learners or teachers, 
Without science, they are likely to overlook useful instructional principle* 
or to embody costly superstitions concerning effective methods. H either 
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art or science is weak, a curriculum is less likely to be effective and 
attractive than if art and science are well combined in the design process. 

The body of this paper has had more to say in a direct way about the 
science than about the art of curriculum design. This is partly because 
we lack even a language of aesthetics for the field of instruction, and in- 
deed for education in general* while we have at least thf* working begin- 
nings of a science of instructional design based on the activities of the past 
dozen or more years in applying principles of psychology and related behav- 
ioral disciplines to the task of education. We speak easily, for example, 
of effectiveness and efficiency with respect to instructional design products, 
but we have difficulty in dealing with elegance as a criterion in in&truction. 

Difficult as it is to define, distinctions between elegant and inelegant 
instruction seem possible. Instruction that is playful, that represents com- 
plex relationships in simple t«rms, that engages learners' attention as a 
result of its internal structu re- -all of these contribute to a sense of ele- 
gance in curriculum; and all are probably related in ways ye^ to be under- 
stood to the more pragmatic goals of effectiveness and efficiency. Thus, 
while focusing in this paper on an emerging science, I have tried here 
and there to point out places where artistic processes and artistic criteria 
seem to play an important role in what I hope will be the coming shape of 
a science of curriculum design. The discipline of curriculum design can 
ill afford the parallel development of two cultures- -scientific and artistic. 
Rather, the two must mteract, with the instructional environment ultimately 
serving as both laboratory and studio, a place where its processes are ob- 
served and where both scientific and artistic standards are brought to bear 
in the interests of education. 
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